1. Field of the Invention
The present invention relates to mechanized manipulators which can fully or partially autonomously operate to perform tasks without direct human control, and in particular, to nonholonomic manipulators which use computerized camera vision to control manipulator movement.
2. Problems in the Art
There are many tasks which by choice or necessity would be completed using basically a robot, instead of a human, or a human utilizing a directly controlled mechanical machine. Examples range from factory assembly line tasks where autonomous robotic manipulation could advantageously decrease labor costs, or eliminate the need for workers to be exposed to dangerous procedures or environments, or extreme procedures such as retrieving or performing some manipulation task outside of a space vessel or at extreme depths in the ocean.
Most mechanized manipulation tasks involve direct human control. For example, a forklift requires a driver to control the steering and movement of the vehicle across the floor, as well as control of the raising and lowering of the fork. Many robotic arms require a human to control movement and positioning of the arm as well as grasping by the arm by direct controls.
There have been a substantial number of attempts, however, of semiautonomousor autonomous manipulation. Many factories utilize preprogrammed robotic arms to complete repetitive tasks in fixed frames of reference. More recently attempts have been made to incorporate what will be called generically "computer vision" striving to allow the machine to detect and recognize items in its field of view and then to operate upon those items accordingly.
A serious shortcoming of all known computer-vision-type manipulation systems is that some sort of pre-known relationship, correlation, or calibration must be made between the major components of the system and the work space in which it operates. For example, U.S. Pat. No. 4,118,730 by inventor Lemelson, entitled "SCANNING APPARATUS AND METHOD" uses a digital camera to attempt to recognize patterns of objects coming into its field of view, and once recognized, allows a manipulator arm to pick them up. However, the entire system is constrained by the fact that the position of the camera, the position and operation of the manipulator arm, the physical response of an end-member to joint rotation, and the algebraic method for assessing the location of items being identified by the camera are all precalibrated. The basic relationships between all of the major components is therefore known and correlated so that the field of view of the camera is pre-calibrated to the actual physical position of the elements within the field of view. Placement accuracy is thereby directly limited by the accuracy of these calibrations.
Another example is U.S. Pat. No. 4,789,940 to Christian, entitled "METHOD AND APPARATUS FOR FILTERING REFLECTIONS FROM DIRECT IMAGES FOR MOBILE ROBOT NAVIGATION". This patent is representative of utilization of a camera to view an area and to keep track of mobile robots in that area so that their movement can be controlled by a control system. While the attempt here is at autonomous control of the robotic vehicles using a type of camera computer vision, a limitation is that the area of movement and the field of view of the camera are and must be carefully precalibrated. In other words, the camera is always in a predetermined known position, and in Christian the floor upon which the robots locomote is carefully calibrated. Movement is directed then in the two dimensional plane of the surface and is accomplished because of these pre-known conditions.
Another attempt places the video camera directly on the end of a robotic arm. Here again, there is an obvious known relationship between the camera and the manipulator end. Even with such a setup, no system is known which has complete autonomous operation, and additional problems of image analysis are incurred with the nonstationary camera.
In direct contrast, if, for example, an antenna on a space ship needed to be repositioned, and the antenna was out of view of the astronauts, to obviate the requirement for the astronauts to go outside the vehicle to accomplish the task, it would be beneficial to have a manipulator means which not only could be moved to any desired position and orientation, but also one which could detect and grasp the antenna on its own without the astronauts' control. This would truly be autonomous manipulation, and such a situation would be incompatible with the high degree of sustained calibration which is usually required when vision is used.
The underpinnings for a vision-based system which produces excellent precision without such calibration are contained in U.S. Pat. No. 4,833,383, by two of the same inventors as the present application and co-owned by the owner of the present application. In U.S. Pat. No. 4,833,383, the means and method for autonomous manipulation between a fixed base and a manipulation arm extending from the fixed base is set forth. While the invention claimed therein does not require that the base necessarily be fixed at all times, the achievement of autonomous manipulation is concerned with situations where the movement of the base does not participate in the actual maneuver of the system.
The patent therefore has a limitation that it covers camera space manipulation only of a certain type of robot, namely one where the internal rotations are algebraically related to the position and orientation of the end-member. This precludes its use with wheeled or "nonholonomic" robots. In other words, the system can control a robotic-type manipulator arm to grasp or position an object held by it with respect to a goal position either within its reach or one which comes within its reach. As can be understood, this limits the work space within which the arm may operate.
Because the manipulator arm of the U.S. Pat. No. 4,833,383 is fixed to the base, even the most complex combination of joints between its connection to the base and its outer effector end can generally be algebraically described; and the required movement of those joints to move the end effector to a desired position can likewise be algebraically predicted and described. This sort of relationship is called a holonomic relationship. No matter what direction or combination of joint movements is achieved, the arm can always be returned to an original starting position simply by returning to the original joint coordinates. This allows complete repeatability of movement. It also allows for algebraic description of such movement.
Holonomic relationships make the autonomous control problem somewhat simpler because finite movements are repeatable. If the manipulator base were on wheels, by contrast, the rotation of the drive wheels and the angle of the steering wheels could be monitored and recorded. Instead of having an algebraic relationship, the relationship between mobile base, manipulator arm and end effector, and a goal location would be differential in nature. The return to an initial rotational position of the drive and steer angle no longer guarantees return to the initial position. Rather, and in contrast with holonomic systems, the entire history of wheel rotation is important.
There is therefore a need to expand the ability of camera space manipulation such as disclosed in the U.S. Pat. No. 4,833,383 from holonomic relationships to nonholonomic relationships, or to the combination of holonomic and nonholonomic relationships. Additionally, there is a need to accomplish nonholonomic camera space manipulation while still retaining precision, accuracy and reliability along with generally "real time" functioning. The complexity of the relationships challenges real time functioning because of the sophisticated and more complex mathematical relationships which in turn would require additional processing time.
It is therefore a principal object of the present invention to provide a nonholonomic camera space manipulation means and method which solves or overcomes the problems and deficiencies in the art.
Another object of the present invention is to provide a means and method as above described which applies to nonholonomic, as well as holonomic relationships.
A still further object of the present invention is to provide a means and method as above described which provides autonomous operation.
A still further object of the present invention is to provide a means and method as above described which allows multiple-degree-of-freedom-of-movement for the manipulator.
Another object of the present invention is to provide a means and method as above described which is operable without calibration or pre-known relationships between any camera means, base means, and work object.
Another object of the present invention is to provide a means and method as above described which is relatively insensitive to imprecision regarding the mathematical models which describe the differential and algebraic kinematics of movement of the manipulator base or the on-board manipulator arm.
A still further object of the present invention is to provide a means and method as above described which does not require merging or direct comparison of fields of view of each camera means.
Still a further feature and advantage of the present invention is to provide a means and method as above described which achieves good real time processing and functioning.
Another object of the present invention is to provide a means and method as above described which is insensitive to visual obscuration of the work piece at or near completion of the manipulation task.
Another object of the present invention is to provide a means and method as above described which eliminates some redundant degrees of freedom of movement by exploiting the available nonholonomic degrees of freedom.
A still further object of the present invention is to provide a means and method as above described which is versatile, precise, reliable, and functions efficiently and quickly for a variety of tasks.
Another object of the present invention is to provide a means and method as above described which has increased utility and flexibility.
These and other objects, features, and advantages of the present invention will become more apparent with reference to the accompanying specification and claims.